Recently, there have been several incidences where firms reported missing MQ/TREX confirm messages.    In each of these incidences, the missing messages were found in the firms’ dead letter queue of their bookkeeping systems.   This ‘alert’ memo is intended to describe the MQM conventions associated with message delivery and the steps one can take to troubleshoot missing messages.  

The CME utilizes IBM’s Websphere MQ application as the infrastructure for delivering all TREX or FIXML based bookkeeping confirmation messages.   One key aspect of the MQ infrastructure is the integrity of the message audit trail.   All messages can be accounted for with respect to delivery and receipt.

Firms define queues in their Queue Manager where they expect to receive messages.   Such messages include electronic trade confirms, give-up API messages, etc.  Firms also define the name and directory location of the dead letter queue where undefined or unexpected messages are deposited.    Additionally, firms define the maximum number of messages to be held in their queue.   Messages that exceed this maximum threshold will be sent to the firm’s defined dead letter queue.

When determining a maximum queue depth, firms should take into consideration the possibility that an application which takes messages off their queues may be down because of a planned or unplanned outage.  CME would have no knowledge of such an outage and could continue to send messages to the firm’s queues.  If a firm defines its queue with a large enough queue depth it can continue to receive messages without overflowing to their dead letter queue.  Defining queues with the maximum depth (640,000 on most platforms) would prevent this problem.

The most recent incidences of firms’ missing messages were caused by the number of messages exceeding the maximum queue depth as defined on their queue.    CME was able to confirm that the messages were sent.   There was no queue depth on the CME side and there were no undeliverable messages in CME’s defined dead letter queue.   On the firms’ side, the messages were ultimately found in the defined dead letter queue in the bookkeeping systems. 

As a precautionary measure, firms should periodically check the maximum message queue depth on their local queues.   As electronic volume and API activity increases, such thresholds need to increase to ensure that all messages are booked in a timely manner.   Additionally, firms should first interrogate their dead letter queues for “top-day” messages when trades are reported missing.  

If you have any questions regarding MQM manager threshold settings or the protocol for interrogating a dead letter queue, please call the CME Clearing Services group at 312-207-2525.

For the Adobe Acrobat version of this advisory...